Embodiments relate to devices, systems, and methods for finding the cause of bugs that are located in a Design under Test (DUT) or in its Verification Environment (VE). The system may be referred to as the Design Verification Environment (DVE). Embodiments may automate portions of the debugging process of simulated hardware designs.
There are different known methods of statistical debugging. However, these methods may suffer from different limitations. For example, methods may be limited to the analysis of code coverage or record vast amounts of run attributes.
Some methods of statistical debugging may be limited to one-dimensional results, i.e., they try to find a single cause for the bug. There are exceptions; they may, however, require powerful learning algorithms. The implication may be the following: either the search space may be very restricted (meaning that many interesting bugs will be overlooked) or heavy statistical tools may be needed, which may typically require over 2000 runs in order to give significant results. This may not be a strong limitation in some software environments, but may be an unrealistic demand in some hardware worlds. For comparison, a typical hardware session may contain only a few hundred runs, and only a small number of these may exhibit a specific failure (sometimes just 1 run).
Some technology also handles noise very badly. Typically, tools return a large set of suspicious code lines or other attributes and the user may still be required to go over many results that have nothing to do with the bug, spending valuable time. This problem may occur mainly due to the law of large numbers: if the search space is big enough, then there may be features that correlate with the bug by mere chance. Some methods may provide a partial remedy to this by using confidence analysis. However, this may only work well for simple binary attributes, whereas the bug may sometimes be caused by a complex cause.